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Program Description 1 



The Lindamood Phoneme Sequencing ® (LiPS®) program (formerly 
called the Auditory Discrimination in Depth ® [ADD] program) is 
designed to teach students the skills they need to decode words 
and to identify individual sounds and blends in words. Initial activi- 
ties engage students in discovering the lip, tongue, and mouth 
actions needed to produce specific sounds. After students are 
able to produce, label, and organize the sounds with their mouths, 
subsequent activities in sequencing, reading, and spelling use the 



oral aspects of sounds to identify and order them within words. 
The program also offers direct instruction in letter patterns, sight 
words, and context clues in reading. UPS ® is designed for emer- 
gent readers in kindergarten through grade 3 or for struggling, 
dyslexic readers. The program is individualized to meet students’ 
needs and is often used with students who have learning disabili- 
ties or difficulties. The version of the program tested here involved 
computer-supported activities. 



Research 2 One study of UPS® that falls within the scope of the Students 
with Learning Disabilities review protocol meets What Works 
Clearinghouse (WWC) evidence standards. The study included 
50 students with learning disabilities from eight to ten years of 
age in three elementary schools in Florida. 3 

Based on this study, the WWC considers the extent of evi- 
dence for UPS® on students with learning disabilities to be small 



for alphabetics, reading fluency, reading comprehension, writing, 
and math. No studies that meet WWC evidence standards with 
or without reservations examined the effectiveness of LiPS® on 
students with learning disabilities in the general reading achieve- 
ment, science, social studies, or progressing in school domains. 



Effectiveness LiPS® was found to have potentially positive effects on alphabetics, reading fluency, and math, no discernible effects on reading 
comprehension, and potentially negative effects on writing for students with learning disabilities. 

1. The descriptive information for this program was obtained from a publicly available source: the program’s website (http://www.lindamoodbell.com, 
downloaded October 2009). The WWC requests developers to review the program description sections for accuracy from their perspective. Further 
verification of the accuracy of the descriptive information for this program is beyond the scope of this review. 

2. The studies in this report were reviewed using WWC Evidence Standards, Version 2.0 (see the WWC Procedures and Standards Handbook, Chapter III). 

3. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 
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Additional program 
information 



Developer and contact 

Developed by Patricia Lindamood and Phyllis Lindamood, the 
LiPS® program is published by Pro-Ed and is available through a 
number of professional distributors and publishers. Address: 416 
Higuera Street, San Luis Obispo, CA 93401. Web: http://www. 
lindamoodbell.com. Telephone: (800) 233-1819. 

Scope of use 

Auditory Discrimination in Depth® was developed in the 
late 1960s and early 1970s. It was revised and renamed the 
Lindamood Phoneme Sequencing® (UPS®) program in 1998. 

The program is frequently offered at centers or clinics, including 
program-endorsed Lindamood-Bell Learning Centers, and is 
available for purchase by the public. 

Teaching 

Teachers work with students in whole class and small group 
activities or in small groups and one-on-one settings to help 



them become aware of the mouth actions that produce speech 
sounds. Instructors help students verify sounds within words 
and teach them to self-correct in reading, spelling, and speech. 
The program developer recommends that instruction last four to 
six months for one hour a day, or four to six weeks for four hours 
a day. Computer-supported activities are available for the pro- 
gram. Lindamood Bell offers UPS® workshops to train teachers, 
but teachers can also learn to administer the program from the 
Lindamood Phoneme Sequencing® Teacher’s Manual. 

Cost 

A kit of materials designed for one-on-one or small-group 
instruction can be purchased for $350. The classroom kit costs 
$541. Kits include a trainer’s manual and all student materials 
(tiles, blocks, colored felts, and picture cards). Some of these 
materials are also sold separately. Information is not available 
on the cost of training for instructors or on how much it costs for 
students to receive instruction at a licensed center. 



Research 



Thirty-one studies reviewed by the WWC investigated the 
effects of LiPS® on students with learning disabilities. One study 
(Torgesen et al., 2001) is a randomized controlled trial that meets 
WWC evidence standards. The remaining 30 studies do not meet 
either WWC evidence standards or eligibility screens. 



Meets evidence standards 

Torgesen et al. (2001) examined the effects of LiPS® using a 
randomized controlled trial involving students with learning 
disabilities between eight and ten years of age from three public 
elementary schools in Florida. All students in the study had 
been identified by school staff as having learning disabilities on 



4. These numbers show the average and range of student-level improvement indices for all findings across the study. 
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Research (continued) the basis of a discrepancy of at least one standard deviation 

between their scores on a standardized test of reading and their 
full-scale score on an intelligence test. Each year for three years, 
the researchers worked with staff from the three elementary 
schools to select a sample of students with learning disabilities 
(20 students per year). The 60 children eligible for the study 
were randomly assigned to one of two groups (30 per group): 
UPS® (formerly Auditory Discrimination in Depth®) or Embedded 
Phonics (an instructional program developed by the authors). 

The interventions were provided to students in two eight-week 
phases: an intensive phase and an application/generalization 
phase. Pretest data were collected two to three weeks prior to 
the start of the interventions, posttest data were collected two to 
three weeks after completion of the eight-week intensive inter- 
vention period, and follow-up data were collected one year and 
two years following the posttest. Ten students were not available 
for the second follow-up data collection, leaving 50 students 



in the analysis sample: 26 students that received UPS® and 24 
students that received Embedded Phonics. 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain as 
small or medium to large (see the WWC Procedures and Stan- 
dards Handbook, Appendix G). The extent of evidence takes into 
account the number of studies and the total sample size across 
the studies that meet WWC evidence standards with or without 
reservations. 5 

The WWC considers the extent of evidence for UPS® to be 
small for students with learning disabilities in the alphabetics, 
reading fluency, reading comprehension, writing, and math 
domains. No studies that meet WWC evidence standards with 
or without reservations examined the effectiveness of LiPS® on 
students with learning disabilities in the general reading achieve- 
ment, science, social studies, or progressing in school domains. 



Effectiveness Findings 

The WWC review of interventions for LiPS® addresses student 
outcomes in nine domains: alphabetics, reading fluency, reading 
comprehension, general reading achievement, writing, math, 
science, social studies, and progressing in school. The study 
included in this report covers five domains: alphabetics (includ- 
ing the constructs of phonological awareness, word attack, and 
letter-word identification), reading fluency, reading comprehen- 
sion, writing, and math. The findings below present the authors’ 
estimates and WWC-calculated estimates of the size and the 
statistical significance of the effects of LiPS® on students with 
learning disabilities. 6 



Alphabetics. Findings from the Torgesen et al. (2001) study 
were based on comparisons of LiPS® students and control group 
students (who participated in an intervention titled Embedded 
Phonics) on eight measures of alphabetics, including four mea- 
sures of phonological awareness, three measures of word attack 
skills, and one measure of letter-word identification. The study 
authors found statistically significant effects favoring the LiPS® 
group for two of the eight measures, including the Lindamood 
Auditory Conceptualization Test and the Word Attack subtest 
from the Woodcock Reading Mastery Test-Revised. WWC analy- 
ses, which were corrected for multiple comparisons, confirmed 
these statistically significant positive findings. Neither the study 



5. The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept— external validity, such as the students’ demographics and the types 
of settings in which studies took place— are not taken into account for the categorization. Information about how the extent of evidence rating was 
determined for L/'PS® is in Appendix A6. 

6. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within class- 
rooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate 
the statistical significance, see WWC Procedures and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, 
Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), a correction for multiple comparisons was needed, so the significance levels 
may differ from those reported in the original study. 
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Effectiveness (continued) 



The WWC found UPS® to 
have potentially positive 
effects on alphabetics, 
reading fluency, and math, 
no discernible effects on 
reading comprehension, and 
potentially negative effects 
on writing for students with 
learning disabilities 



authors nor the WWC found statistically significant effects for the 
other six measures of alphabetics. According to WWC criteria, 
UPS® has potentially positive effects on alphabetics for students 
with learning disabilities. 

Reading fluency. Torgesen et al. (2001) found statistically 
significant effects favoring the UPS® group for two measures 
of reading fluency: the Reading Accuracy and Reading Rate 
subtests from the Gray Oral Reading Test-Ill. WWC analyses 
found neither of these comparisons to be statistically signifi- 
cant; however, the WWC-calculated average effect across the 
two outcomes was large enough to be considered substan- 
tively important according to WWC criteria (that is, an effect 
size of at least 0.25). Thus, the WWC found UPS® to have 
potentially positive effects on reading fluency for students with 
learning disabilities. 

Reading comprehension. Torgesen et al. (2001) found no 
statistically significant effects on either of the two measures of 
reading comprehension: the Passage Comprehension subtest of 
the Woodcock Reading Mastery Test-Revised and the Reading 
Comprehension subtest of the Gray Oral Reading Test-Ill. The 
WWC-calculated average effect size across the two outcomes 
was not large enough to be considered substantively important. 
According to WWC criteria, UPS® has no discernible effects on 
reading comprehension for students with learning disabilities. 



Writing. Torgesen et al. (2001) found, and the WWC confirmed, 
a statistically significant effect favoring the control group on the 
Spelling subtest from the Kaufman Test of Educational Achieve- 
ment. According to WWC criteria, UPS® has potentially negative 
effects on writing for students with learning disabilities. 

Math. Torgesen et al. (2001) found no statistically significant 
effect on the Calculation subtest from the Woodcock-Johnson 
Psycho-Educational Battery-Revised. WWC analyses 
confirmed that there was no statistically significant effect; 
however, the effect on the Calculation subtest was positive 
and large enough to be considered substantively important 
according to WWC criteria. Thus, the WWC concludes that 
UPS ® has potentially positive effects on math for students with 
learning disabilities. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effectiveness 
takes into account four factors: the quality of the research design, 
the statistical significance of the findings, the size of the difference 
between participants in the intervention and the comparison condi- 
tions, and the consistency in findings across studies (see the WWC 
Procedures and Standards Handbook, Appendix E). 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and an 
average improvement index across studies (see WWC Procedures 
and Standards Handbook, Appendix F). The improvement index 
represents the difference between the percentile rank of the aver- 
age student in the intervention condition and the percentile rank of 
the average student in the comparison condition. Unlike the rating 
of effectiveness, the improvement index is entirely based on the 



size of the effect, regardless of the statistical significance of the 
effect, the study design, or the analysis. The improvement index 
can take on values between -50 and +50, with positive numbers 
denoting favorable results for the intervention group. 

Based on one study, the average improvement index for 
alphabetics is +9 percentile points with a range of -4 to +34 
percentile points across eight findings; the average improvement 
index for reading fluency is +17 percentile points based on two 
findings; the average improvement index for reading comprehen- 
sion is +8 percentile points with a range of -5 to +21 percentile 
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The WWC found i/PS® to 
have potentially positive 
effects on alphabetics, 
reading fluency, and math, 
no discernible effects on 
reading comprehension, and 
potentially negative effects 
on writing for students with 
learning disabilities 
(continued) 



points across two findings; the improvement index for writing is 
-22 percentile points based on one finding; and the improvement 
index for math is +12 percentile points based on one finding. 

Summary 

The WWC reviewed 31 studies on UPS® for students with 
learning disabilities. One of these studies meets WWC evidence 
standards; the remaining 30 studies do not meet either WWC 



evidence standards or eligibility screens. Based on the one 
study, the WWC found that UPS® has potentially positive 
effects on alphabetics, reading fluency, and math, no discern- 
ible effects on reading comprehension, and potentially negative 
effects on writing for students with learning disabilities. The 
conclusions presented in this report may change as new 
research emerges. 
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Appendix A1 


Study Characteristics: Torgesen et al., 2001 (randomized controlled trial) 


Characteristic 


Description 


Study citation 


Torgesen, J. K., Alexander, A. W., Wagner, R. K., Rashotte, C. A., Voeller, K., & Conway, T. (2001). Intensive remedial instruction for children with severe reading disabilities: 
Immediate and long-term outcomes from two instructional approaches. Journal of Learning Disabilities, 34(1), 33. 


Participants 


The sample for this study included a total of 60 students between eight and ten years of age. All participants had been identified by school staff as having learning disabilities 
based on a discrepancy of at least one standard deviation between their scores on a standardized test of reading and their full-scale score on an intelligence test. Each 
year for three years, the researchers worked with staff from the three elementary schools to select a sample of students with learning disabilities (20 students per year) 
who met the following criteria: (1) they were identified by their teachers as having serious difficulty in acquiring word-level reading skills, (2) their average standard score on 
two measures of word-level reading was at least 1.5 standard deviations below average, (3) their estimated verbal intelligence was above 75, and (4) they performed below 
minimum required levels for their grade on a measure of phonological awareness. The 60 children eligible for the study were randomly assigned to one of two groups (30 per 
group): LiPS® or Embedded Phonics (an instructional program developed by the authors). The interventions were provided to students in two eight-week phases: an intensive 
phase and an application/generalization phase. Pretest data were collected two to three weeks prior to the start of the interventions, posttest data were collected two to three 
weeks after completion of the eight-week intensive intervention period, and follow-up data were collected one year and two years following the posttest. Ten children were not 
available for second year follow-up data collection, leaving 50 students in the analysis sample — 26 students that received LiPS ® and 24 students that received Embedded 
Phonics. Children in the study had the following characteristics: all were 4th graders, 72% were male, 65% were Caucasian and 35% were African-American, and the average 
full-scale IQ for students in the study was about 96. Additional findings reflecting students’ outcomes at the first and second year follow-ups can be found in Appendices 
A4.1-4.10. 


Setting 


The study was conducted with students with learning disabilities from three public elementary schools in Florida. Students were between eight and ten years of age and 
enrolled in the 4th grade. 


Intervention 


This study involved a comparison of the effectiveness of two interventions: (1) LiPS ® and (2) Embedded Phonics. Both of the interventions provided explicit instruction in 
word-level reading skills; they differed in method of teaching and in relative amount of time spent on instructional activities. LiPS® was provided to students in two phases. 

In the first phase of the intervention, intensive instruction was delivered on a 1 :1 basis for two 50-minute sessions, five days a week, for eight weeks, until a total of 67.5 
hours of instruction had been provided. During this time of intensive instruction, LiPS ® substituted for time the students would normally have spent in their learning disability 
resource room. In the second, less intensive phase of LiPS®, students received one 50-minute lesson per week for eight additional weeks in their learning disability resource 
room, applying skills they had learned during the intensive phase to regular classroom materials. LiPS® placed primary emphasis on building skills in phonemic awareness and 
phonemic decoding with individual words. LiPS ® has three goals: (1) to provide a basis for accurate discriminations among phonemes by teaching the distinctive kinesthetic, 
auditory, and visual (mouth form pictures) features associated with all the common phonemes of the English language; kinesthetic and visual features are taught to help make 
the phoneme more concrete, and to allow children to both hear and feel phonemic contrasts and identities in spoken patterns; (2) to teach children to use their knowledge 
of the distinctive features of phonemes to monitor and represent sequences of sounds in spoken syllables; and (3) to use problem-solving activities to teach children self- 
monitoring skills. 




(continued) 
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Appendix A1 Study Characteristics: Torgesen et al., 2001 (randomized controlled trial) (continued) 



Characteristic 


Description 


Comparison 


Students in the comparison group participated in a competing intervention, developed by the study authors, called Embedded Phonics. This intervention was delivered for the 
same amount of time as UPS® and also taught phonemic awareness and phonemic decoding. However, the Embedded Phonics instruction emphasized application through 
reading meaningful text and recognizing, practicing, and spelling high-frequency sight words. 


Primary outcomes 
and measurement 


The authors assessed students with a battery of tests at the pretest, posttest, one-year follow-up, and two-year follow-up time points. In the domain of alphabetics, phonologi- 
cal awareness was measured by administration of the Lindamood Auditory Conceptualization Test and the Elision subtest, the Non-word Repetition subtest, and the Rapid 
Letter Naming subtest from the Comprehensive Test of Phonological Processes (CTOPP); word attack was measured by administration of the Word Attack subtest of the 
Woodcock Reading Mastery Test-Revised (WRMT-R) and the Phonemic Decoding and Sight Word Efficiency subtests of the Test of the Word Reading Efficiency (TOWRE); 
and letter-word identification was measured by administration of the Word Identification subtest of the WRMT-R. The domain of reading fluency was measured by administra- 
tion of the Reading Accuracy and Reading Rate subtests of the Gray Oral Reading Test-Ill (GORT— III). The domain of reading comprehension was measured by administration 
of the Passage Comprehension subtest of the WRMT-R and the Reading Comprehension subtest of the GORT— III. The domain of writing was measured by administration of 
the Spelling subtest of the Kaufman Test of Educational Achievement. The domain of math was measured by administration of the Calculation subtest from the Woodcock- 
Johnson Psycho-Educational Battery-Revised (WJ-R). Other outcomes were reported in the study but were not included in this report because they were outside the scope of 
the Students with Learning Disabilities review. For a more detailed description of these outcome measures, see Appendices A2.1-2.5. 


Staff/teacher training 


The teachers who administered each program all had at least one year’s experience teaching children with reading disabilities using that method or one very similar to it. The 
teachers who taught UPS® were all drawn from those working at a clinic where the program had been used for the previous five years. 
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Appendix A2.1 Outcome measures for the alphabetics domain 



Outcome measure 


Description 


Phonological awareness construct 

Lindamood Auditory 
Conceptualization Test (LAC) 


The LAC is an individually administered, norm-referenced assessment that measures a child’s ability to perceive and conceptualize speech sounds using a visual 
medium (as cited in Torgesen et al., 2001). 


Elision subtest from the 
Comprehensive Test of 
Phonological Processes (CTOPP) 


The CTOPP Elision subtest measures the child's ability to manipulate sounds in words (as cited in Torgesen et al., 2001). 


Non-word Repetition 
subtest from the CTOPP 


The CTOPP Non-word Repetition subtest measures the ability of a child to combine sounds that are presented orally and to put the separate sounds together to form a 
nonsense word (as cited in Torgesen et al., 2001). 


Rapid Letter Naming 
subtest from the CTOPP 


The CTOPP Rapid Letter Naming subtest measures rate of access to phonological information in long-term memory (as cited in Torgesen et al., 2001). 


Word attack construct 

Word Attack subtest from the 
Woodcock Reading Mastery 
Test-Revised (WRMT-R) 


The WRMT-R Word Attack subtest measures phonemic decoding skills by asking students to read pseudowords. Students are aware that the words are not real (as 
cited in Torgesen et al., 2001). 


Phonemic Decoding subtest 
from the Test of Word Reading 
Efficiency (TOWRE) 


The TOWRE Phonetic Efficiency Decoding subtest measures the number of pronounceable non-printed words that can be accurately decoded within 45 seconds (as 
cited in Torgesen et al., 2001). 


Sight Word Efficiency 
subtest from the TOWRE 


The TOWRE Sight Word Efficiency subtest assesses the number of real printed words that can be accurately identified within 45 seconds (as cited in Torgesen et al., 
2001). 


Letter-word identification construct 

Word Identification subtest 
from the WRMT-R 


The WRMT-R Word Identification subtest measures basic word reading skills and requires the child to read aloud isolated words that range in frequency and difficulty 
(as cited in Torgesen et al., 2001). 



Appendix A2.2 Outcome measures for the reading fluency domain 



Outcome measure Description 



Reading Accuracy subtest The GORT— III Reading Accuracy subtest measures the number of word reading errors that occurred while reading a series of short paragraphs that increase in difficulty (as 

from the Gray Oral Reading cited in Torgesen et al., 2001). 

Test-Ill (GORT— III) 

Reading Rate subtest The GORT— III Reading Rate subtest measures the amount of time taken to read short paragraphs that increase in difficulty (as cited in Torgesen et al., 2001). 

from the GORT-III 
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Appendix A2.3 Outcome measures for the reading comprehension domain 



Outcome measure 


Description 


Passage Comprehension 
subtest from the WRMT-R 


The WRMT-R Passage Comprehension subtest asks students to read silently and fill in missing words in a short paragraph (as cited in Torgesen et at, 2001). 


Reading Comprehension 
subtest from the GORT— III 


The GORT— III Reading Comprehension subtest asks students to read paragraphs and answer five comprehension questions for each paragraph. The questions are read to 
students by the tester (as cited in Torgesen et at, 2001). 


Appendix A2.4 Outcome measures for the writing domain 


Outcome measure 


Description 


Spelling subtest from the 
Kaufman Test of Educational 
Achievement (KTEA) 


The KTEA Spelling subtest is a norm-referenced test consisting of 50 items (as cited in Torgesen et at, 2001). 


Appendix A2.5 Outcome measures for the math domain 


Outcome measure 


Description 


Calculation subtest from 
the Woodcock-Johnson 
Psycho-Educational 
Battery-Revised (WJ-R) 


The WJ-R Calculation subtest measures the subject’s skill in performing mathematical calculations including addition, subtraction, multiplication, and division. The calcula- 
tions involve decimals, fractions, and whole numbers (as cited in Torgesen et al., 2001). 
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Appendix A3.1 Summary of study findings included in the rating for the alphabetics domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(LiPSP- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 



Torgesen et ai., 2001 s 



LAC 


Grade 4 


50 


82.3 

(10.1) 


69.0 

(17.3) 


13.3 


0.93 


Statistically 

significant 


+32 


CTOPP Elision subtest 


Grade 4 


50 


96.4 

(14.3) 


97.9 

(12.8) 


-1.5 


-0.11 


ns 


-4 


CTOPP Non-word 
Repetition subtest 


Grade 4 


50 


102.2 

(16.1) 


103.0 

(19.0) 


-0.8 


-0.04 


ns 


-2 


CTOPP Rapid Letter 
Naming subtest 


Grade 4 


50 


93.3 

(10.3) 


93.1 

(11.5) 


0.2 


0.02 


ns 


+1 


WRMT-R Word 
Attack subtest 


Grade 4 


50 


98.0 

(7.0) 


90.3 

(8.3) 


7.7 


0.99 


Statistically 

significant 


+34 


TOWRE Phonemic 
Decoding subtest 


Grade 4 


50 


84.7 

(4.8) 


83.7 

(5.8) 


1.0 


0.19 


ns 


+7 


TOWRE Sight Word 
Efficiency subtest 


Grade 4 


50 


72.1 

(5.8) 


72.7 

(7.8) 


-0.6 


-0.09 


ns 


-3 


WRMT-R Word 
Identification subtest 


Grade 4 


50 


79.9 

(11.2) 


80.5 

(9.6) 


-0.6 


-0.06 


ns 


-2 


Domain average for alphabetics 9 












0.23 


ns 


+9 



ns = not statistically significant 
LAC = Lindamood Auditory Conceptualization Test 
CTOPP = Comprehensive Test of Phonological Processing 
WRMT-R = Woodcock Reading Mastery Test-Revised 
TOWRE = Test of Word Reading Efficiency 

1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the alphabetics domain. One- and two-year follow-up findings from Torgesen et 
al. (2001) are not included in these ratings, but are reported in Appendices A4.1 and A4.6. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 



(continued) 
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Appendix A3.1 Summary of study findings included in the rating for the alphabetics domain 1 (continued) 



5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple com- 
parisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures 
and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), a correction for 
multiple comparisons was needed, so the significance levels may differ from those reported in the original study. 

9. This row provides the study average, which in this instance, is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The 
average improvement index is calculated from the average effect size. 
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Appendix A3.2 Summary of study findings included in the rating for the reading fluency domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(i/PS®- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 



Torgesen et al., 2001 s 



GORT— III Reading Grade 4 

Accuracy subtest 


50 


93.1 

(12.4) 


87.5 

(13.4) 


5.6 


0.43 


ns 


+17 


GORT— III Reading Grade 4 

Rate subtest 


50 


75.6 

(8.2) 


72.1 

(7.9) 


3.5 


0.43 


ns 


+17 


Domain average for reading fluency 9 










0.43 


ns 


+17 



ns = not statistically significant 
GORT— III = Gray Oral Reading Test-Ill 



1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the reading fluency domain. One- and two-year follow-up findings from Torgesen 
et al. (2001) are not included in these ratings, but are reported in Appendices A4.2 and A4.7. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), no corrections for 
clustering or multiple comparisons were needed. 

9. This row provides the study average, which in this instance, is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The 
average improvement index is calculated from the average effect size. 
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Appendix A3.3 Summary of study findings included in the rating for the reading comprehension domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(i/PS®- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 



WRMT-R Passage 
Comprehension subtest 


Grade 4 


50 


90.2 

(9.0) 


Torgesen et al., 2001 s 

92.0 

(19.8) 


-1.8 


-0.12 


ns 


-5 


GORT— III Reading 
Comprehension subtest 


Grade 4 


50 


91.7 

(10.0) 


86.0 

(10.4) 


5.7 


0.55 


ns 


+21 


Domain average for reading comprehension 9 










0.21 


ns 


+8 



ns = not statistically significant 

WRMT-R = Woodcock Reading Mastery Test-Revised 

GORT— III = Gray Oral Reading Test-Ill 



1. This appendix reports findings considered for the effectiveness rating and the average improvement indices for the reading comprehension domain. One- and two-year follow-up findings from 
Torgesen et al. (2001) are not included in these ratings, but are reported in Appendices A4.3 and A4.8. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), no corrections for 
clustering or multiple comparisons were needed. 

9. This row provides the study average, which in this instance, is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The 
average improvement index is calculated from the average effect size. 
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Appendix A3.4 Summary of study findings included in the rating for the writing domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(A/PS®- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 



Torgesen et ai., 2001 s 



KTEA Spelling subtest 


Grade 4 


50 


76.3 

(5.0) 


80.0 

(7.4) 


-3.7 


-0.58 


Statistically 

significant 


-22 


Domain average for writing 












-0.58 


Statistically 

significant 


-22 



KTEA = Kaufman Test of Educational Achievement 



1 . This appendix reports findings considered for the effectiveness rating and the average improvement indices for the writing domain. One- and two-year follow-up findings from Torgesen et al. 
(2001) are not included in these ratings, but are reported in Appendices A4.4 and A4.9. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), no corrections for 
clustering or multiple comparisons were needed. 
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Appendix A3.5 Summary of study findings included in the rating for the math domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(i/PS®- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 



Torgesen et al., 2001 s 



WJ-R Calculation subtest 


Grade 4 


50 


90.9 

(14.1) 


86.9 

(11.5) 


4.0 


0.30 


ns 


+12 


Domain average for math 












0.30 


ns 


+12 



ns = not statistically significant 

WJ-R = Woodcock-Johnson Psycho-Educational Battery-Revised 



1 . This appendix reports findings considered for the effectiveness rating and the average improvement indices for the math domain. One- and two-year follow-up findings from Torgesen et al. 

(2001) are not included in these ratings, but are reported in Appendices A4.5 and A4.10. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), no corrections for 
clustering or multiple comparisons were needed. 
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Appendix A4.1 



Summary of one-year follow-up findings for the alphabetics domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(Z./PS®- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 



Torgesen et al., 2001 s 



LAC 


Grade 4 


50 


75.4 

(14.0) 


72.0 

(13.9) 


3.4 


0.24 


ns 


+9 


CTOPP Elision subtest 


Grade 4 


50 


93.3 

(13.7) 


94.4 

(14.3) 


-1.1 


-0.08 


ns 


-3 


CTOPP Non-word 
Repetition subtest 


Grade 4 


50 


109.8 

(15.6) 


108.2 

(17.4) 


1.6 


0.10 


ns 


+4 


CTOPP Rapid Letter 
Naming subtest 


Grade 4 


50 


90.0 

(10.6) 


92.4 

(10.2) 


-2.4 


-0.23 


ns 


-9 


WRMT-R Word 
Attack subtest 


Grade 4 


50 


92.3 

(9.3) 


87.0 

(8.9) 


5.3 


0.57 


ns 


+22 


TOWRE Phonemic 
Decoding subtest 


Grade 4 


50 


83.0 

(7.4) 


80.6 

(9.1) 


2.4 


0.29 


ns 


+11 


TOWRE Sight Word 
Efficiency subtest 


Grade 4 


50 


76.9 

(6.4) 


74.4 

(9.6) 


2.5 


0.30 


ns 


+12 


WRMT-R Word 
Identification subtest 


Grade 4 


50 


80.2 

(9.6) 


78.2 

(11.3) 


2.0 


0.19 


ns 


+7 



ns = not statistically significant 
LAC = Lindamood Auditory Conceptualization Test 
CTOPP = Comprehensive Test of Phonological Processes 
WRMT-R = Woodcock Reading Mastery Test-Revised 
TOWRE = Test of Word Reading Efficiency 

1 . This appendix presents one-year follow-up findings for measures that fall in the alphabetics domain. Posttest scores were used for rating purposes and are presented in Appendix A3.1 . 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 



(continued) 
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Appendix A4.1 Summary of one-year follow-up findings for the alphabetics domain 1 (continued) 



7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple com- 
parisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures 
and Standards Flandbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), a correction for 
multiple comparisons was needed, so the significance levels may differ from those reported in the original study. 
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Appendix A4.2 Summary of one-year follow-up findings for the reading fluency domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(i/PS®- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 



Torgesen et ai., 2001 s 



GORT— III Reading 


Grade 4 


50 


97.4 


90.8 


6.6 


0.48 


ns 


+18 


Accuracy subtest 






(12.3) 


(14.8) 










GORT— III Reading 
Rate subtest 


Grade 4 


50 


75.2 

(9.3) 


72.1 

(13.2) 


3.1 


0.27 


ns 


+11 



ns = not statistically significant 
GORT— III = Gray Oral Reading Test-Ill 



1. This appendix presents one-year follow-up findings for measures that fall in the reading fluency domain. Posttest scores were used for rating purposes and are presented in Appendix A3. 2. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), no corrections for 
clustering or multiple comparisons were needed. 
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Appendix A4.3 Summary of one-year follow-up findings for the reading comprehension domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(A/PS®- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 











Torgesen et al., 2001 s 










WRMT-R Passage 


Grade 4 


50 


92.0 


91.5 


0.5 


0.05 


ns 


+2 


Comprehension subtest 






(8.0) 


(10.8) 










GORT— III Reading 
Comprehension subtest 


Grade 4 


50 


96.3 

(10.0) 


88.1 

(12.2) 


8.2 


0.73 


Statistically 

significant 


+27 



ns = not statistically significant 

WRMT-R = Woodcock Reading Mastery Test-Revised 

GORT— III = Gray Oral Reading Test-Ill 



1. This appendix presents one-year follow-up findings for measures that fall in the reading comprehension domain. Posttest scores were used for rating purposes and are presented in Appendix 
A3. 3. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple com- 
parisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures 
and Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), a correction for 
multiple comparisons was needed, so the significance levels may differ from those reported in the original study. 
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Appendix A4.4 Summary of one-year follow-up findings for the writing domain 1 









Authors’ findings from the study 














Mean outcome 
(standard deviation) 2 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Z./PS® Comparison 

group 3 group 


Mean 

difference 4 

(i/PS®- 

comparison) 


Statistical 

Effect significance 6 

size 5 (at a = 0.05) 


Improvement 

index 7 



Torgesen et ai., 2001 s 



KTEA Spelling subtest Grade 4 5 0 75.5 74.0 1.5 0.24 ns +9 

( 5 - 8 ) m 

ns = not statistically significant 

KTEA = Kaufman Test of Educational Achievement 

1. This appendix presents one-year follow-up findings for measures that fall in the writing domain. Posttest scores were used for rating purposes and are presented in Appendix A3.4. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants 
had more similar outcomes. 

3. Each intervention group mean is calculated as the unadjusted control mean plus the WWC-adjusted mean difference. Standard deviations are unadjusted. 

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. 

5. For an explanation of the effect size calculation, see WWC Procedures and Standards Handbook, Appendix B. 

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

8. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple compari- 
sons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of Torgesen et al. (2001), no corrections for 
clustering or multiple comparisons were needed. 
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